1. Field of the Invention
The present invention relates to an image processing device, an image processing method, and an image processing program.
2. Background Art
Recently, techniques of extracting feature information from an image and recognizing an object have been used as a technique of recognizing a general object or recognizing a specific object. As a processing step of efficiently extracting the feature information, an integral image or an integral histogram (integral gradient image) is created.
Since the integral histogram is an integral image for each gradient direction, integral histograms corresponding to the number of gradient directions are necessary.
Recognition techniques using an integral image are described in Japanese Unexamined Patent Application, First Publication No. 2009-87253 (Patent Document 1), “Rapid Object Detection using a Boosted Cascade of Simple Features”, Paul Viola, Michael Jones, Computer Vision and Pattern Recognition 2001 (Non-patent Document 1), and the like.
An adaptive cruise control (ACC) system, a collision avoiding (or reducing) system, a collision warning system, and the like are known as an in-vehicle object recognition device. As such systems, a spread of systems is expected which can recognize multiple types of target objects at a time, such as road traffic signs in addition to vehicles, motorcycles, bicycles, and pedestrians.
A spread of low-cost devices is also expected which can recognize multiple types of target objects using a single in-vehicle camera ECU (Electronic Control Unit). Pattern recognition has been often used for recognition of an object using an in-vehicle camera ECU. An object recognition algorithm using the pattern recognition is constructed by combining classifiers of AdaBoost, SVM (Support Vector Machine), or the like with feature values of Haar-like, HOG (Histograms of Oriented Gradients), or the like.
In the pattern recognition, in order to extract images of target objects from a captured image, an image region (window) from which objects are recognized is set to various sizes and the object recognition algorithm is performed for each window.
In order to efficiently acquire feature values for each window, the technique of creating an integral image or an integral histogram from which a cumulative value of a specific rectangular region in a window is acquired at a high speed is used.
An example of a process flow which is performed by an object recognition unit (for example, a processing unit corresponding to an object recognition unit 13 of a camera ECU 1 shown in FIG. 1) according to the background art will be described below with reference to FIGS. 20 and 21.
FIG. 20 is a flowchart illustrating an example of a process flow which is performed by an object recognition unit according to the background art.
First, the object recognition unit calculates intensity gradients for an image (step S1001).
Examples of calculation of the intensity gradients are shown in FIGS. 16 and 17.
Then, the object recognition unit creates an integral histogram of an arbitrary object recognition region in the entire image region (step S1002).
The process of creating an integral histogram is performed, for example, by any one or more of an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor) and a microcomputer (for example, the same as included in the object recognition unit 13 of the camera ECU 1 shown in FIG. 1).
Here, the total volume of data of the integral histogram becomes a volume of data obtained by increasing the volume of data of the integral image by the number of gradient directions, in that the integral histogram is created for each intensity gradient direction calculated in the process of step S1001.
Then, the object recognition unit transmits data of the created integral histogram to a memory (for example, the same as a memory 23 included in the object recognition unit 13 of the camera ECU 1 shown in FIG. 1) (step S1003).
Subsequently, the object recognition unit performs an object recognition process (a pattern recognition process for object recognition in this example) with the data of integral histograms stored in the memory as input data (step S1004).
As described above, since the volume of data of the integral histogram is large for each gradient direction, it is necessary to design the integral histograms while preventing transmission to the memory in the process of step S1003 and access to the memory in the process of step S1003 from being interrupted.
FIG. 21 is a flowchart illustrating an example of an object recognition process (details of the process of step S1004 in the flowchart shown in FIG. 20) which is performed by the object recognition unit according to the background art.
In this example, a recognition algorithm is constructed by Haar-like feature values and classifiers (such as AdaBoost or SVM).
In this example, a window having a rectangular shape is used.
In the object recognition process (the process of step S1004), the object recognition unit first performs an extraction operation on the regions of the integral histograms stored in the memory with a predetermined coordinate region (window) as a raster scan process (step S1101). In this case, the object recognition unit accesses data of the integral histograms stored in the memory and acquires and uses the data.
Then, the object recognition unit calculates a feature vector as a feature value calculating process (step S1102). In this example, the object recognition unit calculates an HG feature value (vector) as the feature vector.
An example of calculation of the HOG feature value is shown in FIG. 17.
As the HOG feature value, a window is partitioned into small cell units, a rectangular region based on the integral histogram is applied to the cell units, and a cumulative value of the rectangular region is calculated by adding and subtracting the cumulative values at four points. This operation is performed for each gradient direction to obtain a gradient histogram of the cell units.
Then, the object recognition unit recognizes an object (image of the object) which is set as a target in advance based on the calculated HOG feature value as a classification process using classifiers (step S1103).
In this example, real AdaBoost classifiers are used. The real Adaboost classifiers are set to classifiers including weak classifiers having learned the HOG feature values of target objects in advance and including appropriately weak classifiers.
Then, the object recognition unit confirms whether the raster scan process has completed (step S1104), and causes the window to slide over a raster scan region and sequentially repeatedly performs the processes of steps S1101 to S1104 until the raster scan process has completed.
In the raster scan process, for example, after the repeated processes end with the same window scale, the scale of the window or the scanning step are switched (changed), the window is caused to sequentially slide again, and these processes are repeatedly performed until the raster scan process has completed.